Goto

Collaborating Authors

 Everett


Unsupervised Evaluation of Multi-Turn Objective-Driven Interactions

Soroka, Emi, Chopra, Tanmay, Desai, Krish, Lall, Sanjay

arXiv.org Artificial Intelligence

Large language models (LLMs) have seen increasing popularity in enterprise applications where AI agents and humans engage in objective-driven interactions. However, these systems are difficult to evaluate: data may be complex and unlabeled; human annotation is often impractical at scale; custom metrics can monitor for specific errors, but not previously-undetected ones; and LLM judges can produce unreliable results. We introduce the first set of unsupervised metrics for objective-driven interactions, leveraging statistical properties of unlabeled interaction data and using fine-tuned LLMs to adapt to distributional shifts. We develop metrics for labeling user goals, measuring goal completion, and quantifying LLM uncertainty without grounding evaluations in human-generated ideal responses. Our approach is validated on open-domain and task-specific interaction data.


Data-Driven Ergonomic Risk Assessment of Complex Hand-intensive Manufacturing Processes

Krishnan, Anand, Yang, Xingjian, Seth, Utsav, Jeyachandran, Jonathan M., Ahn, Jonathan Y., Gardner, Richard, Pedigo, Samuel F., Adriana, null, Blom-Schieber, null, Banerjee, Ashis G., Manohar, Krithika

arXiv.org Artificial Intelligence

Hand-intensive manufacturing processes, such as composite layup and textile draping, require significant human dexterity to accommodate task complexity. These strenuous hand motions often lead to musculoskeletal disorders and rehabilitation surgeries. We develop a data-driven ergonomic risk assessment system with a special focus on hand and finger activity to better identify and address ergonomic issues related to hand-intensive manufacturing processes. The system comprises a multi-modal sensor testbed to collect and synchronize operator upper body pose, hand pose and applied forces; a Biometric Assessment of Complete Hand (BACH) formulation to measure high-fidelity hand and finger risks; and industry-standard risk scores associated with upper body posture, RULA, and hand activity, HAL. Our findings demonstrate that BACH captures injurious activity with a higher granularity in comparison to the existing metrics. Machine learning models are also used to automate RULA and HAL scoring, and generalize well to unseen participants. Our assessment system, therefore, provides ergonomic interpretability of the manufacturing processes studied, and could be used to mitigate risks through minor workplace optimization and posture corrections.


Titan submersible recovery efforts continue with help of remotely operated vehicle

FOX News

Navy SEAL Jake Zweig responds to the intense search for the missing Titanic submarine on'Fox & Friends.' Efforts to recover the remains of the Titan submersible that suffered a catastrophic implosion near the Titanic wreckage are currently underway, and as of Sunday, had descended to the seafloor for a fourth dive. Last Thursday, the U.S. Coast Guard confirmed that a debris field located about 1,600 feet from the wreckage of the Titanic was in fact that of the missing Titan submersible. The underwater vessel was carrying five men on board when it lost contact with its surface ship about an hour and 45 minutes after descending to the Titanic. South Wellfleet, Massachusetts-based Pelagic Research Services (PRS) was contacted by OceanGate, the company behind Titan, for use of its remotely operated vehicles, or "ROVs," to assist with the search. Pelagic Research Services continues to assist the Transportation Safety Board of Canada, U.S. Coast Guard, and U.S. National Transportation Safety Board with Titan recovery efforts near the Titanic wreckage.


Sam Altman invested $180 million into a company trying to delay death

MIT Technology Review

Altman does not appear on the Forbes billionaires list, but that doesn't mean he isn't extremely wealthy. His wide-ranging investments have included early stakes in companies like Stripe and Airbnb. "I have been an early-stage tech investor in the greatest bull market in history," he says. Now, he is putting his capital to work at a level he calls an "order of magnitude" greater than he could during his Y Combinator days. And he has been concentrating those bets into a few areas of technology he thinks will have the biggest positive impact on human affairs: AI, energy, and anti-aging biotech.


How robots and other tech can make the fight against coronavirus safer

PBS NewsHour

Humans may sometimes regard robots with apprehension or resentment over the increasing automation of labor, but the coronavirus pandemic is showing how the two can work together in new ways that might save lives during a crisis. Around the globe, robots and other technologies, like drones and telehealth devices, are being used in a variety of settings and capacities to assist in the COVID-19 response since there is a level of elevated risk for human workers. Automated devices have delivered meals to quarantined travelers in a Chinese hotel; enforced curfews in Tunisia; scanned visitors for fevers entering a South Korean hospital; monitored patients in a hard-hit Italian city; and tracked social distancing compliance from the skies in a number of cities around the world, including Elizabeth, New Jersey. Many of the technologies were available commercially prior to the coronavirus outbreak, said Texas A&M University professor Robin Murphy, who studies how robots can be deployed during disasters. But now, "they are being used 24/7 and adapted to fit the needs of those using them," Murphy added.


Thank These Riveting Robots for Planes That Don't Fall Apart

WIRED

If you feel like you've taken a beating after spending eight hours on a plane, try spending a shift on the assembly line that rolls out the flying metal tubes you so hate. It takes two humans to install each of the more than 60,000 rivets that hold a Boeing 777 together: one firing the rivet gun, the other holding the steel bucking bar that forces the fastener into place. The benefits of this tiring job include repetitive stress injuries to the arms, back, and shoulders. To improve life on the line, Boeing took a note from Kenny Chesney: no shoulders, no back, no problems. In other words, it brought in robots.


Education Week

AITopics Original Links

Struggling algebra students in the Everett, Wash., school district get help from special tutors who diagnose their weaknesses, tailor instruction to their needs, and provide on-the-spot feedback-all with an inhuman degree of patience. That's inhuman literally: The tutors are computers. Three years ago, the district started employing Cognitive Tutor, a series of computer programs based on artificial intelligence that were developed by researchers from Carnegie-Mellon University in Pittsburgh. The programs provide an alternative form of math instruction to secondary school students who haven't succeeded in regular classrooms. The experience proved so successful that officials in the 20,000-student district have expanded the program.